Optimization using Function Values Only

In this paper, we focus on solving global optimization problems using only the values of the objective function. In particular we explore the Simulated Annealing method, and find improvements using techniques in Derivative Free Optimization methods. The first part of the paper examines the well known Simulated Annealing algorithm. We discuss important theoretical results, and we then demonstrate how each component of Simulated Annealing contributed to solving the global optimization problem invoking only the objective function values. The second part of the paper will be dedicated to techniques from Derivative Free Trust Region method. We discuss how Derivative Free Trust Region determines a second order local optimum. We then propose a method to bypass local optima using Simulated Annealing hill climbing moves. Lastly, we address the shortcomings of Simulated Annealing on continuous optimization problems with strong non-linearity. We present a new method which take into consideration topological information to determine search direction and neighborhood function. We then look at a real life application of the Simulated Annealing method in Appendix A.